Accuracy remains a standard metric for evaluating AI systems, but it offers limited insight into how models arrive at their solutions. In this work, we introduce a benchmark based on brainteasers written in long narrative form to probe more deeply into the types of reasoning strategies that models employ. Brainteasers are well-suited for this goal because they can be solved with multiple approaches, such as a few-step solution that uses a creative insight or a longer solution that uses more brute force. We investigate large language models (LLMs) across multiple layers of reasoning, focusing not only on correctness but also on the quality and creativity of their solutions. We investigate many aspects of the reasoning process: (1) semantic parsing of the brainteasers into precise mathematical competition-style formats; (2) self-correcting solutions based on ground-truth solutions; (3) producing step-bystep sketches of solutions; and (4) making use of hints. We find that LLMs are in many cases able to find creative, insightful solutions to brainteasers, suggesting that they capture some of the capacities needed to solve novel problems in creative ways. Nonetheless, there also remain situations where they rely on brute force, despite the availability of more efficient, creative solutions, highlighting a potential direction for improving LLM reasoning.

Accuracy remains a standard metric for evaluating AI systems, but it offers limited insight into how models arrive at their solutions. In this work, we introduce a benchmark based on brainteasers written in long narrative form to probe more deeply into the types of reasoning strategies that models use. Brainteasers are well-suited for this goal because they can be solved with multiple approaches, such as a few-step solution that uses a creative insight or a longer solution that uses more brute force. We investigate large language models (LLMs) across multiple layers of reasoning, focusing not only on correctness but also on the quality and creativity of their solutions. We investigate many aspects of the reasoning process: (1) semantic parsing of the brainteasers into precise mathematical competition style formats; (2) self-correcting solutions based on gold solutions; (3) producing step-by-step sketches of solutions; and (4) making use of hints. We find that LLMs are in many cases able to find creative, insightful solutions to brainteasers, suggesting that they capture some of the capacities needed to solve novel problems in creative ways. Nonetheless, there also remain situations where they rely on brute force despite the availability of more efficient, creative solutions, highlighting a potential direction for improvement in the reasoning abilities of LLMs.

Accuracy remains a standard metric for evaluating AI systems, but it offers limited insight into how models arrive at their solutions. In this work, we introduce a benchmark based on brainteasers written in long narrative form to probe more deeply into the types of reasoning strategies that models use. Brainteasers are well-suited for this goal because they can be solved with multiple approaches, such as a few-step solution that uses a creative insight or a longer solution that uses more brute force. We investigate large language models (LLMs) across multiple layers of reasoning, focusing not only on correctness but also on the quality and creativity of their solutions. We investigate many aspects of the reasoning process: (1) semantic parsing of the brainteasers into precise mathematical competition style formats; (2) generating solutions from these mathematical forms; (3) self-correcting solutions based on gold solutions; (4) producing step-by-step sketches of solutions; and (5) making use of hints. We find that LLMs are in many cases able to find creative, insightful solutions to brainteasers, suggesting that they capture some of the capacities needed to solve novel problems in creative ways. Nonetheless, there also remain situations where they rely on brute force despite the availability of more efficient, creative solutions, highlighting a potential direction for improvement in the reasoning abilities of LLMs.

In large-scale industrial LLM systems, prompt templates often expand to thousands of tokens as teams iteratively incorporate sections such as task instructions, few-shot examples, and heuristic rules to enhance robustness and coverage. This expansion leads to bloated prompts that are difficult to maintain and incur significant inference latency and serving costs. To address this, we introduce Prompt Compression via Attribution Estimation (ProCut), a flexible, LLM-agnostic, training-free framework that compresses prompts through attribution analysis. ProCut segments prompt templates into semantically meaningful units, quantifies their impact on task performance, and prunes low-utility components. Through extensive experiments on five public benchmark datasets and real-world industrial prompts, we show that ProCut achieves substantial prompt size reductions (78% fewer tokens in production) while maintaining or even slightly improving task performance (up to 62% better than alternative methods). We further introduce an LLM-driven attribution estimator that reduces compression latency by over 50%, and demonstrate that ProCut integrates seamlessly with existing prompt-optimization frameworks to produce concise, high-performing prompts.

There are many different ways to solve mathematical optimization problems. You can use greedy algorithms, constraint programming, mixed integer programming, genetic algorithms, local search, and others. Depending on the size and type of the problem, and the solution quality desired, one technique may work better than the other. This post gives an overview of different heuristics for solving discrete optimization problems. First, I explain the three components you need to describe an optimization problem mathematically.

Exact substring matching is a common task in many software applications. Despite the existence of several algorithms for finding whether or not a pattern string is present in a target string, the most common implementation is a na\"ive, brute force approach. Alternative approaches either do not provide enough of a benefit for the added complexity, or are impractical for modern character sets, e.g., Unicode. We present a new algorithm, Scout, that is straightforward, quick and appropriate for all applications. We also compare the performance characteristics of the Scout algorithm with several others.

In February 2019, the artificial intelligence research lab OpenAI sent shockwaves through the world of computing by releasing the GPT-2 language model. Short for "Generative Pretrained Transformer 2," GPT-2 is able to generate several paragraphs of natural language text -- often impressively realistic and internally coherent -- based on a short prompt. Scarcely a year later, OpenAI has already outdone itself with GPT-3, a new generative language model that is bigger than GPT-2 by orders of magnitude. The largest version of the GPT-3 model has 175 billion parameters, more than 100 times the 1.5 billion parameters of GPT-2. Just like its predecessor GPT-2, GPT-3 was trained on a simple task: given the previous words in a text, predict the next word. This required the model to consume very large datasets of Internet text, such as Common Crawl and Wikipedia, totalling 499 billion tokens (i.e.

A lawyer and hobbyist musician collaborated with a computer programmer to generate every possible 12-note melody in the key of C. The final compilation includes 68.7 billion melodic combinations, which the pair uploaded to the Internet Archive through a Create Commons Zero license, meaning they reserve no rights of ownership to any of them. A lawyer and hobbyist musician partnered with a programmer to create a computer algorithm to generate every 12-note melody possible in the key of C, leading to more than 68.7 billion combinations The project was originally started in 2019 when Damien Riehl, a lawyer and hobbyist musician, and programmer Noah Rubin were having drinks after a cybersecurity event. During the day's presentations Riehl had gotten the idea that it might be possible to'brute force' different combinations of musical notes in the same way that computer hackers brute force different letter and number combinations to crack passwords. At the time, a jury had just ruled against Katie Perry in a lawsuit brough by Flame, a rapper who claimed her chart topper'Dark Horse' had copied a musical fragment from his 2009 song'Joyful Noise.' The team was inspired by computer hackers how use a'brute force' combination technique to crack other people's passwords, and thought a similar approach might be useful for arranging notes into melodic structures Riehl and Rubin originally hoped to have an algorithm come up with every melodic combination of notes possible in western music, using the 88 notes of a standard piano as their starting point.

Artificial Intelligence (AI) is the hot topic of the moment in technology, and the driving force behind most of the big technological breakthroughs of recent years. In fact, with all of the breathless hype we hear about it today, it's easy to forget that AI isn't anything all that new. Throughout the last century, it has moved out of the domain of science fiction and into the real world. The theory and the fundamental computer science which makes it possible has been around for decades. Since the dawn of computing in the early 20th century, scientists and engineers have understood that the eventual aim is to build machines capable of thinking and learning in the way that the human brain – the most sophisticated decision-making system in the known universe – does.

To tackle all of this, Barron's sat down with New Street Research technology infrastructure chief Pierre Ferragu at the investment research boutique's office in Manhattan's Flatiron District. Its open floor plan has a European-design sensibility, yet resembles a small trading operation--a book of Richard Avedon photos makes way for whiteboards covered with remnants of S-curves and math equations as oversize Bloomberg screens flash green and red. Ferragu, 44, joined New Street last year looking for "freedom of thought." He began his career at the Boston Consulting Group, where he advised management teams at media and technology companies; he joined Bernstein in 2008 to make the leap to investment research. But technology is not just a sector in the S&P 500 index--it's also a part of every sector, and Ferragu wanted more freedom to make connections and draw insights from disparate industry groups that can sometimes cause turf wars among analysts.